
Edheatio^), L800.336.5 1 9T (U£,^Can^dd^r J^5d 1 .302.3777 (Int’l), iste@iste. 


Copyright © 2005, ISTE (International Society for Ted 

By David L. Johnson " 


fights reserved 





Copyright © 2005, 15TE '(International Society forTech 


ology in Education), 1.800.336.5 1 9 1 (U.S. & Canada) or 1 .541 .302.3777 (Int’l), iste@iste.org, www.iste.org.AII rights reserved. 





j f i - ' 


tcM 






K{C 


nm 




{ 


A fter decades of research in 
artificial intelligence (AI) 
and cognitive psychology, a 
number of companies have emerged 
that offer intelligent tutor system (ITS) 
software to schools. These systems 
try to mimic the help that a human 
tutor would provide to an individual 
student, something nearly impos- 
sible for teachers to accomplish in the 
classroom. They offer new options to 
identify, remediate, and track students 
in a very individualized manner. 

An ITS can adapt to the student in a 
way that traditional computer-assisted 
instruction (CAI) cannot. It is built on 
the foundations of AI and cognitive 
psychology. AI uses mathematics to 
model human reasoning, employing 
inductive and deductive processes 
to derive novel answers. Cognitive 
psychology explains how humans 
represent and organize knowledge in 
memory. Many of them are based on 
Dr. John R. Andersons adaptive con- 
trol of thought theory, which posits 
that all cognitive functions can be rep- 
resented as a set of production rules. 
{Editor's note: Find a glossary of these 
and other terms on p. 17.) 

A typical ITS diagnoses what feed- 
back a student needs and what prac- 
tice problems should follow, based 
on production rules, not predefined 
branching sequences. Production 
rules enable the ITS to anticipate all 
alternative problem solutions and 
respond to each one. The program 
assesses what the student knows in a 
student skills model. The ITS gener- 
ates a problem, works out a solution, 
compares the solution in real time to 
the student s solution, and performs 


a diagnosis based on differences 
between the two. The system then 
updates the student skills model. This 
customized diagnosis and remediation 
differentiates an ITS from CAI. 

After millions of dollars of funding 
by the National Science Foundation, 
U.S. Department of Education, and the 
U.S. military, the research is paying off 
with commercial ITS products that im- 
prove student achievement. Carnegie 
Learning Inc. and Apangea Learning 
Inc. offer math tutors, and Quantum 
Simulations, Inc. offers chemistry 
tutors. All three companies provide 
intelligent tutoring in different ways, 
and all are backed by solid research. 
Carnegie Learning provides personal 
attention to students in the classroom 
with a complete curriculum, which 
includes teacher training, computer 
tutors for the lab, and textbooks for the 
classroom. Apangea provides personal 
attention to students with an Inter- 
net-based hybrid tutoring model that 
leverages intelligent software for about 
90% of the work and Internet-based 
human tutors for the other 10%. Quan- 
tum Simulations provides personal 
attention to students at home with an 
Internet-based homework tutor that is 
open-ended, allowing students to enter 
any problem their teachers assign. 

Personal Attention in Class 

Carnegie Learning spun out of Carn- 
egie Mellon University as a separate 
company in 1999 and offers a blended 
curriculum for math in which 40% of 
class time is spent in the computer lab 
using the Cognitive Tutor programs 
and 60% is spent in the classroom 
doing class activities. 


Intelligent tutors can mimic human tutors, 

providing some of the same benefits as weii as 
unceasingly patient practice and remediation. 


} 
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Cognitive Tutor Algebra I provides multi- 
ple representations of the problem. A real- 
world word problem Is provided in the top 
left window. A worksheet to oonvert text- 
ual information into quantities and units is 
provided in the bottom left window. A graph 
is provided in the bottom right window, 
as well as a skills meter summarizing the 
student’s mastery of math skills. A solver to 
oonvert information into equations is provided 
in the top right window. Finally, a hint at the 
bottom helps direot the student to properly 
oonvert units In the worksheet. 


Cognitive science researchers at 
Carnegie Mellon teamed with Pitts- 
burgh public school teachers, such 
as former Presidential Teacher of the 
Year Bill Hadley, to develop and refine 
the mathematics curricula over a 20- 
year period. The line of tutors employs 
AI, tracking what students know, tai- 
loring the feedback, and sequencing 
the next problem. 

According to Dr. Steve Ritter, se- 
nior cognitive scientist of the com- 
pany, “What we do is give a complex 
problem and monitor the process the 
student uses to solve the problem. 

We provide multiple representations 
of the problem, linking graphs with 
equations with tables with words. We 
are trying to help students develop 
robust representations to promote 
transfer to real-life applications.” A 
worksheet prompts students to think 
abstractly by converting the problem 
into quantities and units. A graph pro- 
motes visual thinking, and an equa- 
tion solver prompts students to think 
in the language of mathemtics. 

The cognitive tutors employ model 
tracing that detects the strategy the 
student is using to solve the problem. 
They recognize common student er- 
rors and misconceptions and respond 
appropriately. “We provide an inter- 
face where students thinking becomes 
more evident so we can detect skill 
difficulties that they may have,” says 
Ritter. “Merely tracking right or wrong 
answers wont provide that. A student 
may get an answer right, but may have 
taken a circuitous route to get that 


answer because of a misconception. 
The objective is to reveal a students 
thinking.” Feedback enables students 
to self-correct. A Skillometer visually 
summarizes the student s mastery of 
math skills. The bars increase in size 
and turn gold on mastery. 

In the classroom, teachers can spend 
more time on concepts after consult- 
ing a Teachers Toolkit that provides 
progress reports for individuals and the 
whole class and a Skills Alert Report 
listing students who are having difficul- 
ty mastering a skill. Carnegie provides 
activities that use problems similar to 
those used in the lab. 

Mary Ann Stine, director of cur- 
riculum and instruction for the Ever- 
ett (Washington) Public Schools, has 
witnessed Cognitive Tutors success. 

At Cascade High School, they placed 
students who had already failed Al- 
gebra in a class using the Cognitive 
Tutor. “After Cognitive Tutor, they had 
a high pass rate of 80-90%,” Stine re- 
ports. They surveyed the students, and 
“80% of those students — this is a total 
of 150 students — said if we could do 
math like this, we would go on to the 
next level,” says Stine. “It s very power- 
ful to hear them talk about how this 


really makes sense to them.” 

Jillian Whaley is a math teacher 
at Stephen Decatur High School in 
Worcester County, Maryland, who 
uses Cognitive Tutor Algebra I with 
9th through 12th graders. Not only 
does the self pacing promote stu- 
dent-to-teacher collaboration, Whal- 
ley says, the software also promotes 
student-to-student collaboration. 

“The kids help each other. A lot of 
times you learn best from your peers. 
There s a lot of math talk going on in 
the lab; you wouldn’t believe it!” The 
personalized attention seems to work. 
“It s absolutely amazing to watch their 
progress,” she continues. 

Personal Attention out of Class 

Based on a lO-year research project 
undertaken by the U.S. Air Force Re- 
search Laboratory and funded in part 
by the Heinz Endowments, Apangea 
Learning was founded in 2002. The 
company sells a hybrid tutoring sys- 
tem for math called SmartHelp that 
combines intelligent software with 
online human tutors for 5th through 
12th graders. SmartHelp promotes a 
structured approach to solving math 
problems. The intelligent tutoring 
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Glossary 

Adaptive Control of Thought (ACT) 

This cognitive theory developed by Dr. John R. Anderson of Carnegie Mellon University states 
that learners aequire eognitive skill by develeping proeedural knowledge (knowing how to do 
things). Learners initially preeess ineeming stimuli as deelarative knowledge (knowing what 
the werld is like). Learners proeess deelarative knewledge into procedural knewledge while 
selving a preblem. Thereafter, repeatedly performing the proeedure strengthens it, allowing 
the learner to execute the preeedure in the future with little eenseious theught. 


Artificial Intelligence 

Al mimies human theught and eognitive proeesses to solve eemplex preblems. 


Cognitive Tutor Authoring Tools 

These are a suite of software tools that facilitate the development of computer-based tutors. 
Carnegie Mellen University ereated the tools, and the Pittsburgh Seienee of Learning Center 
(http://www.learnlab.erg) uses them for learning seienee experiments. 


Computer-Assisted Instruction (CAI) 

This type of application employs conditional branching after assessing the student’s response, 
either looping the student through built-in remediation or advancing the student to the next 
preblem. Develepers eonstruetthe branehes in advanee. In eontrast, intelligent tuter systems 
(ITS) braneh students en the fly. Likewise, CAI typieally deesn’t reeognize miseoneeptiens or 
diagnose student errors and tailor remediation based on these errers as an ITS does. 


Consistency Rules 

These Al rules, develeped by Quantum Simulations Inc., check a student’s steps in selving 
a preblem fer eensisteney with an applieable prineiple er eoneept in the subjeet demain, 
permitting intelligent tutor systems te effer epen-ended questions. 


Intelligent Tutor Systems (ITS) 

These applieatiens use Al software teehnolegies and eognitive psyehelogy medels to provide 
ene-en-one instruetien. They evaluate student performanee, assess the student knewledge 
and skills, previde instruetional feedbaek, and seleet apprepriate next exereises for the 
student. 


Model Tracing 

This is the preeess ef eomparing student respenses te an expert system. An expert system 
eonsists ef a set ef preduetion rules fer the domain of knowledge. Every time the student 
takes an aetien, the tutor compares it to a set of actions generated by the expert system. If 
the student aetien matehes one ef the aetions predieted by the expert system, the student 
aetion is eensidered eorreet. Medel traeing ean identify miseeneeptiens and typieal student 
errors and provide appropriate feedbaek te the student. 


Production Rules 

Preduetion rules represent knowledge in eognitive psyeholegy expressed as a set of IF-THEN 
rules. Fer example, IF yeu want to start the ear, THEN insert the key on the right side of the 
steering wheel and turn the key away frem yeu. 


Student Skills Model 

This represents the proeedural skills the tuter pereeives that the student has. The tutor 
compares the student skills model to the expert system te identify differenees and te seleet 
the next problem for the student to solve. 


Thank you, 



Please join ISTE in 
welcoming these new 
Corporate Members as they 
support ISTE s efforts 
to improve teaching and 
learning by advancing the 
effective use of technology 
in education. 

AutoSkill 
International, Inc. 

FirstClass 


Learn more about this outstanding 
group of corporations and how 
they can help you at 
www.iste.org/istelOO/. 


International Society for Technology in Education 



Partners in Educational Technology Leadership 
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^ Available Intelligent Tutor Systems 

1 

Apangea Learning Inc. ■ http://www.apangea.( 

Dom 

Modules 

Prioe 

SmartHelp 

$48 

SmartHelp LIVE! (includes human tutors online) 

$60 

Professional Development 

No charge 

Teacher’s Toolkit 

No charge 

Note: SmartHelp and SmartHelp LIVE! offer 27 tutorial math courses for Grades 5-1 2. Prices vary, depending on concurrent seat 
licenses and usage. Annual subscriptions include human tutors online. Alternatively schools may use their own tutors/teachers online. 

Carnegie Learning Inc. ■ http://www.carnegielearning.conn 

Modules 

Prioe 

Cognitive Tutor Algebra I 

$65 

Cognitive Tutor Geometry 

$65 

Cognitive Tutor Algebra II 

$65 

Cognitive Tutor Integrated Mathematics 

$65 

Bridge to Algebra (new course) 

$65 

Cognitive Tutor Textbook 

Included 

Teacher’s Toolkit 

Included 

Teacher training 

Included (regional onsite 
training is $2,500/day 
for up to 25 teachers) 

Professional development 

$750/teacher/day 

Note: Prices vary depending on number of students. $65 is for a typical implementation (50-1 00 students) and includes textbooks, 
training, and support. 

Quantunn Sinnulations Inc. ■ http://www.quantunnsinnulations.conn 

Modules 

Prioe 

Complete Bundle (Chemistry/Math) 

$43 

Chemistry Bundle 

$31 

Mathematics Bundle 

$21 

Individual Tutor Modules 

$9.95 

Note: Quantity discounts are available based on volume and multiple-year subscriptions. Chemistry modules include measurement, 
the elements, ionic compound formulas, mathematics of chemical formulas, equation balancing, oxidation numbers, chemical bonding, 
chemical reactions, and stoichiometry. Mathematics modules include measurement, ratio and proportion, percentages, scientific 
notation, and metric units. 


Join thO dobato! Take our online poll based on this issue’s Point/Counterpoint topic 
at http://www.iste.org/ll. Take it a step further and send us your take on the subject and you 
may see your words and photo in an upcoming edition. 




ACT NOW 
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software handles 90%-95% of the 
work and for the remainder, passes 
the student to a human tutor onsite 
or on the Web. The software has been 
used successfully for remediation in 
after-school programs, study hall pro- 
grams, and evening tutoring at home. 

The Air Force studied the best prac- 
tices of human tutors as they helped 
students with the process of solving 
problems. From this process meth- 
odology, the Air Force developed a 
line of software for math, science, 
and writing and tested their approach 
on 30,000 public school students 
throughout the United States. Apan- 
gea licensed the Air Force technology 
and developed a unique combination 
of intelligent software and human tu- 
tors for its 27 courses in mathematics. 
Using a seven-step process, the tutors 
help build critical skills that teach stu- 
dents how to think about math. 

Apangeas vision is to leverage tech- 
nology to make tutoring affordable 
and available to every student. “We 
decided to provide large-scale tutor- 


ing programs to schools, where in 
the past they could only afford to do 
very small-scale tutoring programs,” 
says Louis Piconi, co-founder and 
CEO. “We do everything from the 
assessment at the front end to gen- 
erating individual learning pathways 
to delivering the tutoring. After each 
lesson, kids work through progres- 
sively more difficult problems as they 
show increasing mastery. As they are 
going through these problems, the tu- 
tor is looking over their shoulder and 
interacting with them based on their 
demonstrated proficiency for each in- 
dividual skill.” 

SmartHelp starts by giving each 
student a placement exam. The soft- 
ware recommends an individual 
learning pathway for each student that 
requires the teacher s approval. Each 
module consists of individual 10- to 
18 -minute lessons. Each lesson ends 
with a quiz to measure understanding 
before allowing students to progress 
to the skills-based problem solving 
session. Struggling students receive 


tutor- generated feedback as they make 
mistakes. This feedback is shaped by 
student proficiency ratings that the 
Tutor continuously adjusts. When the 
system can no longer help a student, a 
human tutor is flagged to provide ad- 
ditional assistance. 

SmartHelp generates real-time re- 
ports on student progress. The reports 
are accessible on the Internet minutes 
after lessons are completed, permit- 
ting the administration to track stu- 
dents compared to standards and to 
adjust the curriculum and classroom 
instruction if necessary. 

Since 2003, Dennis Johnson, super- 
intendent of the Cornell School Dis- 
trict in Coraopolis, Pennsylvania, has 
used SmartHelp in the school districts 
tutorial programs. He implemented 
these programs because nearly 25% 
of the students in the Cornell School 
District were performing below state 
standards for math and reading. 

Since then, the district has seen 
significant progress and has met per- 
Tutors continued on page 23. 



SmartHelp 
provides real-time 
reports to distriot 
superintendents 
who traok student 
progress against 
standards. 
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Tutors continued from page 19. 
formance standards on the Pennsyl- 
vania System of School Assessments 
exam for the first time in four years. 
Approximately 32% of eighth grad- 
ers tested during the spring of 2004 
scored at or above proficient levels, 
up from 28% in 2003. 

Johnsons district sends progress 
reports to parents every two weeks. 
“That s key,” he says. “You Ve got to 
have the capability to generate those 
reports. You have to keep parents and 
guardians informed. Its a valuable 
tool for us to determine if a child has 
mastered a concept or needs another 
intervention.” 

Personal Attention at Home 

After $4 million of funding from 
the U.S. Department of Education, 
National Science Foundation, and 
the National Institutes of Health, 
Quantum Simulations released its 
line of chemistry and science- related 
mathematics tutors in 2002. These 
are the first and only Al-based tutors 
for these disciplines. Building on the 
model tracing capabilities developed 
by the researchers at Carnegie Mellon 
University, Quantum innovated and 
created a tutor that allows students to 
enter their own problems — a switch 
from a typical tutor, in which the 
computer generates the problems. 

The Quantum Tutors are particu- 
larly effective in helping students dur- 
ing private study when no teacher is 
present to answer their questions. The 
Tutors show students how to work 
their problems one step at a time, ex- 
plaining each step in detail and allow- 
ing students to ask questions before 
moving on to the next step. This facili- 
tates self- explanation, which in turn 
promotes understanding. 

Chemist Dr. Benny Johnson teamed 
up with his former high school chem- 
istry teacher. Dale Holder, to form the 
company. Holder has taught chemis- 
try in Kentucky for 40 years and has 
won numerous teaching awards. They 


couldnt use a conventional CAI ap- 
proach because the computer would 
not know in advance what chemistry 
problems the students were going to 
use or what mistakes they were go- 
ing to make. Therefore, the company 
developed a breakthrough innovation, 
called Consistency Rules and received 
a methodology patent. 

The company is nearly finished cre- 
ating assessment tools that will permit 
teachers to create their own assess- 
ment questions and students to take 
the assessments online and receive 
real-time feedback. The Quantum As- 
sessment Advisors will instantly grade 
the assessment, award partial credit, 
and for missed questions, transfer the 
question data back to the Quantum 
Tutor for remediation. The Advisor 
will provide data to teachers and ad- 
ministrators summarized by school 
and by district, with drill-down detail 
for individual students. 

Kevin Willis teaches 12th-grade 
chemistry in the Carmichaels Area 
Junior/Senior High School, near Pitts- 
burgh, Pennsylvania. He uses the tutor 
for homework assignments, “as a take- 
home teacher,” he says. “A lot of times 
the students don t even know what 
questions to ask because its so foreign to 
them. The tutor helps them ask the ques- 
tions they need whenever they get stuck.” 

Willis also uses the tutor in the 
computer lab for review sessions with 
another set of homework problems. 
Rather than working through all the 
problems on the board, he requires the 
students to work through the problems 
on their own with the tutor. “They can 
ask specific questions of the tutor and 
get feedback,” says Willis. “If they have 
further questions, I can come around 
and help them. The value in review and 
individualized instruction really helps.” 

Willis has been pleased with the re- 
sults. “The experience that we had with 
the Balancing Equations tutor has been 
amazing. I wasn t a skeptic, because I 
thought it could work, but I have to 
admit I was surprised with the results 


because there was such a large increase 
for the students who used the tutor.” 

In a trial, one class used the tutor and 
another did not. Both classes received 
almost identical scores on a pretest and 
both showed gains between the pretest 
and posttest. However, the tutor group 
scored 12.8 points better than the con- 
trol group — one whole letter grade. “The 
difference was astounding,” says Willis. 

Future Directions 

ITS software is extremely time con- 
suming to develop, with estimates of 
200-1,000 hours of development time 
required to produce one hour of in- 
struction. Cognitive Tutor Authoring 
Tools developed by Carnegie Mellon 
University show promise, reducing 
development time to 23 hours per 
each hour of instruction. The tools 
are included in the research the Pitts- 
burgh Science of Learning Center will 
conduct on how people learn. The 
Center was established in 2004 with a 
$25 million grant from the National 
Science Foundation to Carnegie Mel- 
lon University, the University of Pitts- 
burgh, and Carnegie Learning. 

Providing personalized attention to 
students through computer technol- 
ogy shows great promise in a way that 
is almost impossible for teachers to 
accomplish on their own. Whether in 
the classroom, after class, or at home, 
ITS technology has proven to be very 
successful in remediation and improv- 
ing standards-based test scores. The 
technology works for all students, 
whether they are excelling or strug- 
gling, because students can progress 
at their own pace. ITS software tools 
help schools achieve results that 
someday will approach what a human 
tutor can provide. 

David L Johnson is a doctoral 
student in the Learning Tech- 
nologies program at the Col- 
lege of Education and Human 
Development, University of 
Minnesota. 
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